Mining User-Generated Repair Instructions from Automotive Web Communities

The objective of this research was to automatically extract user-generated repair instructions from large amounts of web data. An artifact has been created that classifies a web post as containing a repair instruction or not. Methods from Natural Language Processing are used to transform the unstructured textual information from a web post into a set of numerical features that can be further processed by different Machine Learning Algorithms. The main contribution of this research lies in the design and prototypical implementation of these features. The evaluation shows that the created artifact can accurately distinguish posts containing repair instructions from other posts e.g. containing problem reports. With such a solution, a company can save a lot of time and money that was previously necessary to perform this classification task manually

Supporting Human Cognitive Writing Processes: Towards a Taxonomy of Writing Support Systems

In the field of natural language processing (NLP), advances in transformer architectures and large-scale language models have led to a plethora of designs and research on a new class of information systems (IS) called writing support systems, which help users plan, write, and revise their texts. Despite the growing interest in writing support systems in research, there needs to be more common knowledge about the different design elements of writing support systems. Our goal is, therefore, to develop a taxonomy to classify writing support systems into three main categories (technology, task/structure, and user). We evaluated and refined our taxonomy with seven interviewees with domain expertise, identified three clusters in the reviewed literature, and derived five archetypes of writing support system applications based on our categorization. Finally, we formulate a new research agenda to guide researchers in the development and evaluation of writing support systems

Unleashing the Potential of Argument Mining for IS Research: A Systematic Review and Research Agenda

Argument mining (AM) represents the unique use of natural language processing (NLP) techniques to extract arguments from unstructured data automatically. Despite expanding on commonly used NLP techniques, such as sentiment analysis, AM has hardly been applied in information systems (IS) research yet. Consequentially, knowledge about the potentials for the usage of AM on IS use cases appears to be still limited. First, we introduce AM and its current usage in fields beyond IS. To address this research gap, we conducted a systematic literature review on IS literature to identify IS use cases that can potentially be extended with AM. We develop eleven text-based IS research topics that provide structure and context to the use cases and their AM potentials. Finally, we formulate a novel research agenda to guide both researchers and practitioners to design, compare and evaluate the use of AM for text-based applications and research streams in IS

A taxonomy for deriving business insights from user-generated content

Deriving business insights from user-generated content (UGC) is a widely investigated phenomenon in information systems (IS) research. Due to its unstructured nature and technical constraints, UGC is still underutilized as a data source in research and practice. Using recent advancements in machine learning research, especially large language models (LLMs), IS researchers can possibly derive these insights more effectively. To guide and further understand the usage of these techniques, we develop a taxonomy that provides an overview of business insights derived from UGC. The taxonomy helps both practitioners and researchers identify, design, compare and evaluate the use of UGC in this IS context. Finally, we showcase an LLM-supported demo application that derives novel business insights and apply the taxonomy to it. In doing so, we show exemplary how LLMs can be used to develop new or extend existing NLP applications in the realm of IS

Design and Evaluation of an AI-based Learning System to Foster Students\u27 Structural and Persuasive Writing in Law Courses

Structured and persuasive writing is essential for effective communication, convincing readers of argument validity, and inspiring action. However, studies indicate a decline in students\u27 proficiency in this area. This decline poses challenges in disciplines like law, where success relies on structured and persuasive writing skills. To address these issues, we present the results of our design science research project to develop an AI-based learning system that helps students learn legal writing. Our results from two different experiments with 104 students demonstrate the usefulness of our fully working AI-based learning system to support law students independent of a human instructor, time, and location. In addition to providing our embedded software artifact, we document our evaluated design knowledge as a design theory. Thus, we provide the first step toward a nascent design theory for the development of AI-based learning systems for legal writing

Designing for Conversational System Trustworthiness: The Impact of Model Transparency on Trust and Task Performance

Designing for system trustworthiness promises to address challenges of opaqueness and uncertainty introduced through Machine Learning (ML)-based systems by allowing users to understand and interpret systems’ underlying working mechanisms. However, empirical exploration of trustworthiness measures and their effectiveness is scarce and inconclusive. We investigated how varying model confidence (70% versus 90%) and making confidence levels transparent to the user (explanatory statement versus no explanatory statement) may influence perceptions of trust and performance in an information retrieval task assisted by a conversational system. In a field experiment with 104 users, our findings indicate that neither model confidence nor transparency seem to impact trust in the conversational system. However, users’ task performance is positively influenced by both transparency and trust in the system. While this study considers the complex interplay of system trustworthiness, trust, and subsequent behavioral outcomes, our results call into question the relation between system trustworthiness and user trust

Design and Evaluation of an Adaptive Empathy Learning Tool

Empathy is an elementary skill for daily interactions and for professional communication, agile teamwork and successful leadership and thus elementary for educational curricula. However, educational organizations face difficulties in providing the boundary conditions necessary for their students to develop empathy skills due to the lack of individual support in traditional large-scale and growing distance-learning scenarios. Drawing on cognitive dissonance theory, we propose an adaptive empathy learning tool that helps students develop their ability to react to other people’s observed experiences through individual feedback in large-scale or distance learning scenarios. Based on a design science research project, we propose a set of design principles and instantiate and evaluate them with our prototype Eva in an online experiment with 65 students. The findings suggest that an adaptive empathy learning tool that follows our design principles is a promising approach to individually support students in their ability to react to other people’s observed abilities in traditional learning scenarios